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DETAILED ACTION 

Drawings 

1 . The drawings are objected to because in reference sign "8" in Figure 1 , "A/C 
converter" should be - A/D converter - as discussed in page 32, line 4 of the 
specification. 

Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the immediate 
prior version of the sheet, even if only one figure is being amended. The figure or figure 
number of an amended drawing should not be labeled as "amended." If a drawing figure 
is to be canceled, the appropriate figure must be removed from the replacement sheet, 
and where necessary, the remaining figures must be renumbered and appropriate 
changes made to the brief description of the several views of the drawings for 
consistency. Additional replacement sheets may be necessary to show the renumbering 
of the remaining figures. Each drawing sheet submitted after the filing date of an 
application must be labeled in the top margin as either "Replacement Sheet" or "New 
Sheet" pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, 
the applicant will be notified and informed of any required corrective action in the next 
Office action. The objection to the drawings will not be held in abeyance. 
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Specification 

2. Applicant is reminded of the proper language and format for an abstract of the 
disclosure. 

The abstract should be in narrative form and generally limited to a single paragraph on a separate 
sheet within the range of 50 to 1 50 words. It is important that the abstract not exceed 1 50 words in length 
since the space provided for the abstract on the computer tape used by the printer is limited. The form 
and legal phraseology often used in patent claims, such as "means" and "said," should be avoided. The 
abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need 
for consulting the full patent text for details. 

The language should be clear and concise and should not repeat information given in the title. It 
should avoid using phrases which can be implied, such as, "The disclosure concerns," "The disclosure 
defined by this invention," "The disclosure describes," etc. 



3. The abstract of the disclosure is objected to because the abstract exceeds 150 
words in length. 



Correction is required. See MPEP § 608.01 (b). 



4. The disclosure is objected to because of the following informalities: 
Page 1, line 4: The - CROSS REFERENCE TO RELATED APPLICATIONS - section 
of the specification is required to be place above the "BACKGROUND OF THE 
INVENTION" to inform of any related applications, in this case the Foreign Priority 
application "Japan 009285/2003 01/17/2003". 

Appropriate correction is required. 
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Claim Objections 

5. Claims 1 , 3-5, 7-8, 10, 13, 15-17, 19-20, and 22-26 are objected to because of 
the following informalities: 

Claims 1,13, and 23-26, lines 3-4 respectively: "processing the image" should be - 
processing an image --. 

Claims 1,13, and 23-26, lines 4-5 respectively: "storing the processed" should be - 
storing a processed -. 

Claims 1 , 23, and 25, lines 7-8 respectively: "extracting the face" should be -- extracting 
a face -. 

Claim 1, line 10: "executing the process" should be - executing a process -. 
Claims 1, 23, and 25, lines 10-11 respectively: "inferring the attributes" should be - 
inferring attributes -. 

Claims 1, 23, and 25, lines 14-15 respectively: "adjusting the image" should be - 
adjusting image -. 

Claims 3 and 15, line 3 respectively: "containing the" should be -- containing a -. 
Claims 4 and 16, line 5 respectively: "specifying the size of should be - specifying a 
size of --. 

Claims 5 and 17, line 2 respectively: "the focal length" should be - a focal length -. 
Claims 7 and 19, line 3 respectively: "the deletion" should be - a deletion --. 
Claims 8 and 20, line 3 respectively: "the operation" should be ~ an operation --. 
Claims 10 and 22, line 3 respectively: "the direction of the face" should be - a direction 
of the face --. 



Application/Control Number: 10/758,905 Page 5 

Art Unit: 2624 

Claims 13, 24, and 26, line 9 respectively: "of the face image" should be - of a face 
image --. 

Claims 13, 24, and 26, lines 10-11 respectively: "and the information required for 
adjusting the" should be - and information required for adjusting an -. 
Claims 13, 24, and 26, lines 11-12 respectively: "with the identification" should be - with 
identification — . 

Appropriate correction is required. 

Claim Rejections - 35 USC § 101 

Claims 23 and 24 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. Claims 23 and 24 are drawn to 
functional descriptive material NOT claimed as residing on a computer readable 
medium. MPEP 21 06. IV. B. 1(a) (Functional Descriptive Material) states: 

"Data structures not claimed as embodied in a computer-readable medium are descriptive 
material per se and are not statutory because they are not capable of causing functional change in the 
computer." 

"Such claimed data structures do not define any structural or functional interrelationships between 
the data structure and other claimed aspects of the invention which permit the data structure's 
functionality to be realized." 

Claims 23 and 24, while defining a "program to be executed by an image pickup 
device", do not define a "computer-readable medium" and is thus non-statutory for that 
reasons. A program can range from paper on which the program is written, to a 
program simply contemplated and memorized by a person. The examiner suggests 
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amending the claim to embody the program on "computer-readable medium" in order to 
make the claim statutory. 

"In contrast, a claimed computer-readable medium encoded with the data structure defines structural and 
functional interrelationships between the data structure and the computer software and hardware 
components which permit the data structure's functionality to be realized, and is thus statutory." - MPEP 

21 06. IV. B. 1(a) 

Claim Rejections -35USC§112 

6. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

7. Claims 1-26 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

Re Claim 1, line 12, claim 13, line 8, claims 23 and 25, line 11, claims 24 and 26, lines 
8-9 respectively: The limitation "based on the feature amounts" in line 12 lacks clear 
antecedent basis. It is suggested to be - based on a feature amounts --. 

Claims 2, 4-5, and 10-1 1 are dependent upon claim 1 . 

Re Claim 3, line 3, claim 12, line 10, claim 15, line 2 respectively: The limitation 
"producing the link information" in line 3 lacks clear antecedent basis. It is suggested to 
be - producing a link information -. 
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Re Claims 6 and 18, line 3 respectively: The limitation "designating the range" in line 3 
lacks clear antecedent basis. It is suggested to be - designating a range --. 

Re Claims 7 and 19, lines 2 and 8 respectively: The limitation "second operating unit" 
renders this claim indefinite because it is unclear what the first operating unit is since 
the base claim 1 (or base claim 13) does not consist of a first operating unit. It is 
suggested to be - first operating unit -• 

Re Claims 8 and 20, lines 2 and 7-8: The limitation "third operating unit" renders this 
claim indefinite because it is unclear what the first and second operating units are since 
the base claim 1 (or base claim 13) does not consist of a first and second operating unit. 
It is suggested to be - first operating unit --. 

Re Claims 9 and 21, lines 2 and 8: The limitation "fourth operating unit" renders this 
claim indefinite because it is unclear what the first, second and third operating units are 
since the base claim 1 (or base claim 1 3) does not consist of a first, second, and third 
operating unit. It is suggested to be - first operating unit -. 

Re Claim 13, line 23: The limitation "object estimated by the inference part" renders this 
claim indefinite because it is unclear when the inference part made an object estimation. 
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Therefore, the limitation "an inference part for inferring the object" in line 16 of claim 13 
is suggested to be - an inference part for estimating the object --. 

Re Claim 14, line 6: The limitation "the image pickup operation" in line 6 lacks clear 
antecedent basis. It is suggested to be - an image pickup operation -. 

Claims 16-17, and 22 are dependent upon claim 13. 

Appropriate correction is required. 

Claim Rejections - 35 (JSC § 102 

8. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

9. Claims 1-3, 5-9, 11-12, 23, and 25 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Center (US 2002/01 1 3862 A1 ). 

Re Claim 1 : Center discloses an image pickup device / videoconferencing system (see 
paragraph [0002], [0027], [0018], abstract) comprising an image pickup unit / camera 
including a lens / lens of camera and an image sensor / camera sensors such as zoom 
and focus sensors, and a control unit / computer system connected to camera for 
processing the image picked up by the image pickup unit and storing the processed 
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image in an internal memory / memory of camera or computer system RAM or a 
predetermined storage medium (see Fig. 1, paragraphs [0018], [0003], and [0027], the 
computer is connected to the camera and they interact in real time since this is video 
conferencing), wherein the control unit includes a face image extraction part / locate 
face for extracting the face image contained in the image picked up by the image pickup 
unit (see Fig. 1, paragraphs [0018] and [0019]), an inference part / evaluate image 
quality for executing the process of inferring / evaluating the attributes / information or 
characteristics of a person constituting an object based on the feature amounts / color 
or motion analysis in an image area including the face image extracted (see paragraphs 
[0020], [0025], and [0004]), an image pickup conditions adjusting part / adjust and 
control the camera for adjusting the image pickup conditions / brightness, contrast and 
color balance for the image pickup unit / camera based on the result of inference / frame 
evaluation in the inference part (see Fig. 1, paragraph [0020]), and an information 
processing part / processing pass for storing in selected one of the memory and the 
storage medium / computer system RAM the image obtained / current frame or adjusted 
frame under the image pickup conditions adjusted by the image pickup conditions 
adjusting part (see paragraphs [0020], [0026], and [0027], the computer stores a current 
frame and does the evaluation of this information and then uses and produces an 
adjustment based on the evaluation and stores this adjustment frame). 
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Re Claim 2: Center further discloses an inference part / evaluate image quality for 
executing the inference / evaluation of at least one of the race, age and sex as the 
attributes / range of appearance (see [0028], lines 4-9, [0048], the evaluation includes 
any combination of the template matching, motion detection, background differencing, 
and color analysis, in this case the template matching searches for a face using 
templates that represent the range of appearance of the different types of faces which 
are known as eigenfaces and these eigenfaces typically use race and sex). 

Re Claim 3: Center further discloses wherein the information processing part / 
processing pass includes a part for producing the link information / hypothesis 
containing the position / location of the face image extracted by the face image 
extraction part / locate face and the inference information / evaluation information 
obtained by the inference process / evaluation executed by the inference part, and 
wherein the link information / hypothesis is stored in selected one of the memory and 
the storage medium / computer system RAM together with the image / current frame 
picked up by the image pickup unit (see paragraph [0048], lines 5-9). 

Re Claim 5: Center further discloses wherein the control unit / computer system 
includes the focal length adjusting part for adjusting the focal length of a lens of the 
image pickup unit / camera in accordance with the result of extraction by the face image 
extraction part / locate face (see paragraph [0004], lines 5-8, once the face is extracted 
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and detected and the computer system adjusts the camera to automatically focus based 
on the detection and evaluation results). 

Re Claim 6: Center further discloses comprising a first operating unit / results Pyramid 
of visual detector for designating the range / location and size of extracting a face 
image, wherein the face image extraction part / locate face includes a part for limiting 
the face image extraction area in the image picked up by the image pickup unit / camera 
in accordance with the designating operation of the first operating unit (see paragraphs 
[0050] and [0004], with the range specified by the visual detector the camera is adjusted 
by focusing and zooming to the correct location and size of the face). 

Re Claim 7 [as best understood by the Examiner]: Center further discloses comprising a 
second operating unit / color analysis for designating the deletion / remove and 
substitute of the result / skin probability of extracting a predetermined part / area of skin 
of the face image extracted, wherein the face image extraction part / locate face 
includes a part for updating the result of extracting the face image in accordance with 
the designating operation of the second operating unit (see paragraphs [0004], [0046], 
and [0047], an update of the face area parameters of the color analysis is accomplished 
by the deletion of skin probability in the lookup table followed by the addition or 
substitution from the user and it is these parameters which update and adjust the 
brightness, contrast and color balance of the image). 
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Re Claim 8 [as best understood by the Examiner]: Center further discloses comprising a 
third operating unit for performing the operation of correcting / adjusting the inference 
information / information or characteristics of a person obtained by the inference 
process of the inference part / evaluate image quality, wherein the information 
processing part / processing pass includes a part for correcting the inference 
information in accordance with the correcting operation of the third operating unit (see 
paragraphs [0026] and [0020], abstract, the processing passes which are the template 
matching, motion detection, background differencing, and color analysis adjust the 
information of the face, paragraphs [0046] and [0047], the color analysis pass looks up 
and adjusts the skin area table values U and V for proper probability calculations, the U 
and V values are related to the color information as seen in [0027] lines 17-19). 

Re Claim 9 [as best understood by the Examiner]: Center further discloses comprising a 
fourth operating unit / camera parameter unit for correcting the image pickup conditions 
/ brightness, contrast and color balance adjusted by the image pickup conditions 
adjusting part / adjust and control the camera, wherein the image pickup conditions 
adjusting part includes a part for readjusting the image pickup conditions in accordance 
with the correcting operation of the fourth operating unit (see Fig. 1 , paragraph [0004], 
lines 5-1 3, paragraph [0020]). 

Re Claim 1 1 : Center further discloses comprising a feature amount / skin color 
probability storage part for storing the feature amount of the face image already 
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extracted, wherein the face image extraction part includes a specified image extraction 
part / locate face for extracting an image area including the feature amount of the 
specified face image stored in the feature amount storage part from the image picked 
up by the image pickup unit (see paragraph [0046], lines 22-27, a skin color probability 
is stored in colorPyramid after the face is extracted from the current incoming image 
and after it goes through the color analysis processing pass). 

Re Claim 12 [as best understood by the Examiner]: Center further discloses comprising 
an object storage part / computer system RAM for storing the feature amount / skin tone 
color probability of the face image of a specified object / user or users, wherein the 
information processing part / processing pass compares / evaluates for modification the 
feature amount / skin tone color probability of the face image extracted by the face 
image extraction part / locate face with the feature amount / skin color probability stored 
in the object storage part / computer system RAM, so that in the case where the 
comparing process shows that the extracted face image is that of the specified object, 
the link information / hypothesis containing the inference information / information or 
characteristics of a person obtained by the inference process of the inference part / 
evaluate image quality and the information / U and V values for identifying the specified 
object / current face image is produced and stored in selected one of the memory and 
the storage medium / computer system RAM together with the image picked up / current 
image by the image pickup unit (see Fig. 1, [0046], lines 22-27, [0047], lines 1-5, [0048], 
lines 5-9, [0050]). 
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As to claims 23 and 25, the claims are the corresponding program (35 U.S.C. 
101 issue) and method to system claim 1 respectively. The discussions are addressed 
with regard to claim 1 . 

Claim Rejections - 35 USC § 103 

10. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

1 1 . Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Center 
as applied to claim 1 above. The teachings of Center have been discussed above. 
Re Claim 10: Center discloses the information processing part / processing pass 
includes a part for determining the direction / location of the face of an object in the 
image based on the result of extraction of the image stored in selected one of the 
memory and the storage medium by the face image extraction part / locate face, and a 
part for rotating / tilt movable camera the image in such a manner that the face direction 
conforms with a predetermined reference direction in the case where the determined 
face direction is different from the reference direction (see [0024], the wide-angle 
movable camera may tilt). 
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Although Center doesn't specifically recite the limitation of determining the 
direction of the face and conforming the determined direction with the reference 
direction, it would be obvious to one of ordinary skill in the art at the time the invention 
was made to include this limitation in the processing pass in order to actually have 
Center's wide-angle movable camera tilt in the appropriate direction to actually optimize 
the face location detector (Center, paragraph [0024]). 

12. Claims 13-15, 17-22, 24 and 26 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Center in view of Soriano ("Making saturated facial images useful 
again", a Non-patent literature reference supplied by the Applicant). 
Re Claim 13: Center discloses an image pickup device / videoconferencing system (see 
paragraph [0002], [0027], [0018], abstract) comprising an image pickup unit / camera 
including a lens / lens of a camera and an image sensor / camera sensors such as 
zoom and focus sensors, and a control unit / computer system connected to the camera 
for processing the image picked up by the image pickup unit and storing the processed 
image in selected one of an internal memory / memory of camera or computer system 
RAM and a predetermined storage medium (see Fig. 1, paragraphs [0018], [0003], and 
[0027], the computer is connected to the camera and they interact in real time since this 
is video conferencing), wherein the control unit includes a registration part / skin tone 
color probability lookup table for holding the feature amount / skin tone color probability 
of the face image of each of a predetermined number of objects / users (see [0047], 
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lines 1-5, a predetermined number of objects are a multiple number of users) and the 
information required for adjusting the optimum image pickup conditions / optimum 
brightness, contrast and color balancing in correspondence with the identification 
information / skin tone color unique to the object, a face image extraction part / locate 
face for extracting the face image contained in the image picked up by the image pickup 
unit (see Fig. 1 paragraphs [0018] and [0019]), an inference part / evaluate image quality 
for inferring / evaluating the object by comparing the feature amount / color or motion 
analysis of the face image extracted by the face image extraction part with the 
information registered in the registration part (see paragraphs [0020], [0025], and 
[0004]), an image pickup conditions adjusting part / adjust and control the camera for 
adjusting the image pickup conditions / brightness, contrast and color balance for the 
image pickup unit using the registered information of the object estimated / evaluated by 
the inference part (see Fig. 1, paragraph [0020]), and an information processing part / 
processing pass for storing in selected one of the memory and the storage medium / 
computer system RAM the image obtained under the image pickup conditions adjusted 
by the image pickup conditions adjusting part (see paragraph [0020], [0026], and [0027], 
the computer stores a current frame and does the evaluation of this information and 
then uses and produces an adjustment based on the evaluation and stores this 
adjustment frame). 

However, Center fails to disclose or fairly suggest specifically that a registration 
part holds the information required for adjusting the optimum image pickup conditions in 
correspondence with the identification information unique to the object. 
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Soriano discloses information / ideal condition value required for adjusting the 
optimum image pickup conditions / brightness, contrast, and color balancing (Center 
discloses these pickup conditions) in correspondence with the identification information / 
skin tone color (Center discloses this information) unique to the object (Soriano, Section 
2.2 Color Correction by RGB eigenfaces, Section 3.1 Physics-Based Face Database, 
abstract, lines 6-8, the pickup conditions are adjusted to the ideal condition values to 
recover color information in facial images taken under non-ideal conditions). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Center's device using Soriano's teachings by 
including the information required for adjusting the pickup conditions to Center's 
registration part in order to recover color information in facial images taken under non- 
ideal conditions (Soriano, abstract, lines 6-8). 

As to claim 15, the discussions are addressed with respect to claim 3. 
As to claim 17, the discussions are addressed with respect to claim 5. 
As to claim 18, the discussions are addressed with respect to claim 6. 
As to claim 19, the discussions are addressed with respect to claim 7. 
As to claim 20, the discussions are addressed with respect to claim 8. 
As to claim 21, the discussions are addressed with respect to claim 9. 
As to claim 22, the discussions are addressed with respect to claim 10. 
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As to claims 24 and 26, the claims are the corresponding program (35 U.SC 
101 issue) and method to system claim 13 respectively. The discussions are addressed 
with regard to claim 13. 

Re Claim 14 [as best understood by the Examiner]: Center further discloses the control 
unit / computer system includes a part / automatic gain control (AGC) for receiving the 
input of the information required (taught by Soriano above) for adjusting the optimum 
image pickup conditions / optimum brightness, contrast, and color balancing and the 
identification information / skin tone color of the object in response to the image pickup 
operation / processing pass of a predetermined object for registration in the registration 
part / skin tone color probability lookup table, and storing / computer system RAM the 
input information in the registration part together with the face image of the object (see 
[0003]-[0004], [0046]-[0048], and [0050]). 

Although Center's AGC is part of the camera and not part of the control 
unit or computer system, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to have an AGC type component built into a computer 
and controlled by the computer in order for the color correction or adjustment to meet its 
highest quality by having the automatic gain control type component be dynamically 
adjusted by the computers inputs. 
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13. Claims 4 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Center as applied to claims 1 and 13 above, and further in view of Aoki (EP 1 158786 
A2). The teachings of Center have been discussed above. 

However, as recited in claims 4 and 16 , Center fails to disclose or fairly suggest a 
distance recognition part. 

Aoki discloses comprising a distance recognition part / distance calculator for 
recognizing the distance to an object / user's face, wherein the face image extraction 
part / face detector includes a part for specifying the size / size filling the monitor of the 
face image to be extracted, based on the result of recognition by the distance 
recognition part (see paragraphs [0132]-[0134]). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Center's device using Aoki's teachings by 
including the distance recognition part for resizing in order to locate and display the face 
fully onto the monitor (Aoki, [01 32]). 

Conclusion 

14. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Cohen-Sola et al discloses a conferencing system and method 
for automatic determination of preset positions corresponding to participants in video- 
mediated communications; Toyama et al discloses a system and method for 
automatically adjusting gaze and head orientation for video conferencing; Aoki et al 
discloses a data transmission method apparatus using same, and data transmission 
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system; Yan discloses a system and method for biometrics-based facial feature 
extraction; Kinjo discloses an image processing method; Steinberg et al discloses a 
digital image processing using face detection information; Beek et al discloses a face 
imaging system for recordal and automated identity confirmation; Sugimoto discloses an 
image characteristic portion extraction method; Imagawa et al discloses a face detection 
device; Okano et al discloses a system for identifying individuals; Owada discloses an 
image pickup device and automatic focus detection method. 

1 5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Bernard Krasnic whose telephone number is (571 ) 270- 
1357. The examiner can normally be reached on Mon-Thur 9:00am-3:00pm and every 
other Friday 9:00am-3:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on (571) 272-7429. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
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USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Bernard Krasnic 
March 7, 2007 




